Testing frameworks for client/server applications, for example, remote applications such as web applications, typically have a graphical user interface (GUI) in which a user can define test cases. Alternatively, some testing frameworks may provide some Application Programming Interface (API) which can be used to define the test cases. An example of a framework with a GUI is a testing tool like WebKing™ from Parasoft® Corp., in which the user can set up test cases within the graphical interface presented by the program.
An example of a framework that provides an API would be the Java library HttpUnit in which the user can define test cases by writing code that refers to the HttpUnit API. In some cases, a testing tool may have both a GUI and an API with which to set up test cases. The tools that use both methodologies usually use a programming language or API that can only be run within the context of the tool.
Typically, two primary aspects of a test case are defined for a remote application. One is the part that defines what requests to make to the web server. The other is the part that determines the validation done on responses sent back by the server.
The main advantage of using a testing tool with a GUI is that the tool typically gives the user easy ways to define the requests to make to the web server. For example, the test tool may allow the user to record the actions taken in a web browser and store them so that they can be “replayed” at a later date. This is usually quite easy when done in this way. However, the user may want more advanced control over the test case than the GUI may provide, like the ability to loop over a portion of the test case or the ability to conditionally run a certain part of the test. These are tasks that are quite easily done using test cases defined in source code but are difficult to provide within a GUI.
Testing tools usually also provide functionality for validating the responses returned by the server being tested. For example, the tool may automatically set up validation tests that attempt to determine that the web server is responding properly. However, due to the dynamic nature of most server applications, it can be quite difficult for the tool to create tests that do not report false positives.
Alternatively, the tool may allow the user to manually set up validation tests on specific portions of the response. This eliminates false positives, and may be quite easy to use if it resides within a graphical interface, but it likely does not provide the level of control a user might need in order to set up the required test. Both of these examples are cases where using a GUI is easier but using test cases defined in source code provides more flexibility and less noisy tests.
While source code is more customizable, the main disadvantages are that it often requires a user to learn a non-standard programming language; it requires understanding of programming; and it is more tedious to write.
Therefore, there is a need for a software testing tool that more easily and efficiently generates test cases defined in computer source code to test client/server applications.